Method and system for segmenting an image

ABSTRACT

The invention pertains to a method for segmenting an image from a sequence of video images into a foreground and a background, said image being composed of pixels, the method comprising: assigning an initial foreground probability to each of said pixels; assigning a set of probability propagation coefficients to each of said pixels; and applying a global optimizer to the initial foreground probabilities of said pixels to classify each of said pixels as a foreground pixel or a background pixel, to obtain a definitive foreground map; wherein said global optimizer classifies each processed pixel in function of the initial foreground probability of said processed pixel and the initial foreground probability of neighboring pixels, the relative weight of the initial foreground probability of neighboring pixels being determined by the probability propagation coefficients of said neighboring pixels.

FIELD OF INVENTION

The present invention relates to the field of video image processing, inparticular foreground detection in video images.

BACKGROUND

Certain visual applications require separating the foreground from thebackground in images of a video feed in order to put the foregroundsubject into another context. Classical applications of such aseparation and recombination are the representation of the TV weatherperson in front of a virtual meteorological map, commercials that appearto be filmed in an exotic scenery, super heroes in movies that appear tobe present in artificial sceneries or flying in the sky. Suchrepresentations have in common that their human subjects are recorded ina studio and that the background is replaced by a new one. The procedureis relatively easy if the original background can be controlled, or ifit is fully known to the video processing unit. This is the case for theexamples above, which are recorded in a specially designed studio.

The present invention deals with the problems that arise when the abovementioned technique has to be used with an arbitrary background, whichhas to be detected or learned by the video processing unit.

SUMMARY

According to an aspect of the present invention, there is provided amethod for segmenting an image from a sequence of video images into aforeground and a background, the image being composed of pixels, themethod comprising: assigning an initial foreground probability to eachof the pixels; assigning a set of probability propagation coefficientsto each of the pixels; and applying a global optimizer to the initialforeground probabilities of the pixels to classify each of the pixels asa foreground pixel or a background pixel, to obtain a definitiveforeground map; wherein the global optimizer classifies each processedpixel in function of the initial foreground probability of the processedpixel and the initial foreground probability of neighboring pixels, therelative weight of the initial foreground probability of neighboringpixels being determined by the probability propagation coefficients ofthe neighboring pixels.

It is an advantage of the present invention that it obtains a highefficiency by combining a “soft” classification based on a probabilisticmodel which may consist of a single pass, with an iterative edge-awarefilter that makes sure that the foreground cut out is consistent withthe actual edges of the objects represented in the image. The assigningof a foreground probability is typically based on a heuristic. Thedetection of edges is abstracted to an assessment of the degree to whichneighboring pixels or regions are correlated, which determines thedegree to which they should be allowed to influence each other'sclassification as foreground or background. Each pixel is thus assigneda set of probability propagation coefficients, which includes estimatesof the probability that an edge is present in the image between thepixel of interest and one or more neighboring pixels.

In an embodiment of the method of the present invention, the assigningof the foreground probability comprises applying at least a firstheuristic probability assignment algorithm and a second heuristicprobability assignment algorithm, and determining the foregroundprobability by combining a result of the first heuristic probabilityassignment algorithm with a result of the second heuristic probabilityassignment algorithm.

Accordingly, the accuracy and robustness of the method of the inventionmay be improved by combining two or more different heuristic models, toobtain a better estimate of whether a given pixel is a foreground pixelor not.

In a specific embodiment, one of the first heuristic probabilityassignment algorithm and the second heuristic probability assignmentalgorithm comprises comparing the pixels to a background color model.

This embodiment takes advantage of any a priori knowledge about thebackground that may be available to the system, but it may additionallyor alternatively rely on knowledge about the background that is acquiredduring the performance of the method of the invention. Acquiredknowledge includes knowledge about the same image contributed by othersources of foreground probability information, in particular anotherheuristic, and/or information about previous images in the same videostream, for which the foreground/background segmentation has alreadytaken place. By a judicious incorporation of acquired information, theneed for a preliminary training phase can be reduced or eliminated.

In a specific embodiment, one of the first heuristic probabilityassignment algorithm and the second heuristic probability assignmentalgorithm comprises applying a human body model.

This embodiment takes advantage of the fact that the object of interestin the most common foreground extraction applications is a human being,typically a human being standing or sitting in front of a camera.

In an embodiment, the method of the present invention further comprisesupdating a parameter of at least one of the first heuristic probabilityallocation algorithm and the second heuristic probability allocationalgorithm on the basis of the definitive foreground map.

It is an advantage of this embodiment that the heuristic algorithms usedfor the soft classification step are dynamically updated and thereforebecome gradually more accurate. The system employing the method thusbecomes self-learning to a certain extent, resulting in more efficientand/or more accurate processing of subsequent video images, based on theassumption that the background does not change significantly from oneimage to the next.

In an embodiment, the method of the present invention further comprisesa post-processing step for removing allocation artifacts.

The additional post-processing step may improve the visual quality ofthe resulting image, by eliminating parts of the alleged foregroundwhich are unlikely to be actual foreground (i.e., “false positives” ofthe foreground detection process), for instance because they have ashape and/or dimensions that are uncharacteristic for foregroundobjects. Conversely, parts of alleged background which are unlikely tobe actual background (i.e., “false negatives” of the foregrounddetection process) may be restored to foreground status.

In a specific embodiment, the post-processing step comprises applyingmorphological operators.

In a specific embodiment, the post-processing step comprises applying aconnected-component algorithm.

In an embodiment of the method according to the present invention, theglobal optimizer is an iteratively applied bilateral filter

According to another aspect of the invention, there is provided acomputer program product which, when executed, causes the computer toperform the method according to the invention.

According to another aspect of the invention, there is provided a systemfor segmenting an image from a sequence of video images into aforeground and a background, the image being composed of pixels, thesystem comprising: an input interface for obtaining the image; anassignment engine for assigning an initial foreground probability toeach of the pixels; a contour detector for assigning a set ofprobability propagation coefficients to each of the pixels; and a globaloptimizer for operating on the initial foreground probabilities of thepixels to create a definitive foreground map by classifying each of thepixels as a foreground pixel or a background pixel; and an outputinterface to provide the definitive foreground map; wherein the globaloptimizer is adapted to classify each processed pixel in function of theinitial foreground probability of the processed pixel and the initialforeground probability of neighboring pixels, the relative weight of theinitial foreground probability of neighboring pixels being determined bythe probability propagation coefficients of the neighboring pixels.

In an embodiment of the system according to the invention, theassignment engine is adapted to apply at least a first heuristicprobability assignment algorithm and a second heuristic probabilityassignment algorithm, and to determine the foreground probability bycombining a result of the first heuristic probability assignmentalgorithm with a result of the second heuristic probability assignmentalgorithm.

In a specific embodiment, one of the first heuristic probabilityassignment algorithm and the second heuristic probability assignmentalgorithm comprises comparing the pixels to a background color model.

In a specific embodiment, one of the first heuristic probabilityassignment algorithm and the second heuristic probability assignmentalgorithm comprises applying a human body model.

In an embodiment, the system according to the present invention isfurther adapted to update a parameter of at least one of the firstheuristic probability allocation algorithm and the second heuristicprobability allocation algorithm on the basis of the definitiveforeground map.

In an embodiment, the system according to the invention furthercomprises a post-processor for removing allocation artifacts from thedefinitive foreground map.

In an embodiment of the system according to the present invention, theglobal optimizer is an iterative bilateral filter

The advantages of the embodiments of the computer program product andthe system according to the invention are the same, mutatis mutandis, asthose of the corresponding embodiments of the method according to theinvention.

BRIEF DESCRIPTION OF THE FIGURES

Some embodiments of apparatus and/or methods in accordance withembodiments of the present invention are now described, by way ofexample only, and with reference to the accompanying drawings, in which:

FIG. 1 comprises a flow chart of an embodiment of the method accordingto the invention;

FIG. 2 comprises a block diagram of an embodiment of the systemaccording to the invention;

FIG. 3 comprises a flow chart of a further embodiment of the methodaccording to the invention; and

FIG. 4 comprises a flow chart of a further embodiment of the methodaccording to the invention.

DESCRIPTION OF EMBODIMENTS

Embodiments of the present invention may be used, without limitation, tosegment the foreground for a home application, like immersive chattingor video conferences.

It is a purpose of embodiments of the present invention to detect theforeground, i.e. regions of interest in the image that must not bereplaced by the chosen substitute background. This detection has tosegment the image along the actual edges of the object of interest, i.e.it must be accurate, and work correctly under varying circumstances,i.e. it must be robust.

Known foreground detection techniques suffer drawbacks including a needfor an initialization phase (in particular to train the backgroundmodel), and a sensitivity to even small changes in lighting conditionsor camera position.

The present invention is based inter alia on the insight that it isadvantageous not to base the foreground estimation solely on a colormodel, but rather on one or more heuristic probabilities indicating thata pixel might or might not be foreground, combining these probabilitieswhere applicable, including object boundary information and eventuallysolving the global segmentation problem.

The heuristics may include imposing a body model (optionally based on aface-detector), using depth information, using motion, using skin-color,foreground color histogram, etc., on top of having a traditionalbackground color and/or an edge model. All heuristics yield aforeground-probability map. These maps are combined into one roughpre-estimate.

Since no boundary information of the original image is included yet, theabove-mentioned rough estimate will not accurately follow the edges ofobjects. It may be considered as a soft classification of the pixels.

According to the invention, it is advantageous to apply a filteringalgorithm to equal out the probabilities of individual regions boundedby edges in such way that each contiguous region is either considered tobe wholly background or wholly foreground. This filtering algorithm,which may be compared to “coloring between the lines” comprises solvinga global optimization problem posed by a data term, following from theprobability maps, and a smoothing term, following from the objectboundaries. The filtering algorithm may be an iterative algorithm.

It is advantageous to further improve the quality of the output of thefiltering algorithm by applying a post-processing step.

The foreground/background map obtained through the steps recited above,may be fed back to one or more heuristic models, to update theparameters of these models. Also the output foreground estimation itselfmay act as a heuristic on its own for the foreground estimation of thenext input video frame, because foreground of subsequent frames tend tobe similar.

FIG. 1 comprises a flow chart of an embodiment of the method accordingto the invention.

In a first step 110, a soft classification of all pixels is performed,which comes down to assigning to each pixel a certain probability thatthe pixel belongs to the foreground.

In a second step 120, a set of probability propagation coefficients isassigned to each of the pixels. This is a generalized way to detectedges in the images; it could thus be considered a “soft” edge detection(herein also referred to as “contour detection”). The probabilitypropagation coefficients represent the degree to which neighboringpixels or regions are correlated, which in the context of the presentinvention determines the degree to which they are allowed to influenceeach other's classification as foreground or background.

In an embodiment, the set of probability propagation coefficientsincludes four probability values for each pixel, representing therespective probabilities that an object edge is present between thepixel of interest and each of the four surrounding pixels (not countingdiagonally adjacent pixels). In a different embodiment, the set ofprobability propagation coefficients includes eight probability valuesfor each pixel (including diagonally adjacent pixels). Otherconfigurations are also possible.

In an embodiment, the probability propagation coefficients are limitedto the values 0 and 1, which results in a “hard” edge detection. If“hard” edge detection is used, the image is de facto segmented incontiguous regions in this step 120.

Edge detection (either “hard” or “soft”) can take place in any of theknown ways, including the detection of significant steps in the spatialdistribution of the intensity or the chroma of the pixels of the image.

In a third step 130, a global optimizer is applied to smoothen out theprobability map, using the detected edges as boundaries to obtain asegmentation into homogenous foreground objects and background objects.

The global optimizer may be applied under the form of a single-pass oran iterative algorithm, e.g. a bilateral filter. Accordingly, acompletion test may be provided at step 140, which determines whetheranother pass of the algorithm is required or not before the definitiveforeground/background map is outputted 150. The completion test mayinclude a test relating to the quality of the segmentation that has beenobtained, but it may also be a simple counter that enforces apredetermined number of algorithm passes.

The end result, or any intermediate result, of the illustrated processmay be used in a feedback loop to update 145 the parameters of theprobability assignment algorithms.

The order in which the steps are presented is without significance,unless the above description implies that one step cannot take placebefore the completion of another step. For example, the second step 120may freely be performed before, during, or after the first step 110,because these two steps are functionally independent.

FIG. 2 comprises a block diagram of an embodiment of the system 200according to the invention.

The block diagram schematically shows an input interface 210 forobtaining the image to be processed. The interface may be present as aphysical and/or a functional interface. The image may for example beobtained over a network, by reading a physical medium, or by invoking asoftware function call.

The obtained image to be processed is forwarded to the assignment engine220 which serves to assign a foreground probability to each pixel of theimage.

The obtained image to be processed is also forwarded to the contourdetector 230 which serves to assign probability propagation coefficientsas described above, which will further on allow segmenting the imageinto a plurality of contiguous regions.

Without loss of generality, the forwarding of the image to theassignment engine 220 and the contour detector 230 is illustrated as aparallel connection.

The information resulting from the operation of the assignment engine220 and the contour detector 230 is passed on to the global optimizer240 which serves to create a definitive foreground/background map byclassifying each region as a foreground region or a background region.The global optimizer 240 may be an iterative bilateral filter.

Finally, the definitive foreground/background map is made available tothe system user via an output interface 250, which may again be presentas a physical and/or a functional interface. In particular, the outputimage may be made available through a network, written on a physicalmedium, or held for retrieval by a subsequent software function call.

The functions of the various elements shown in the figures, includingany functional blocks described “interfaces” or “engines”, may beprovided through the use of dedicated hardware as well as hardwarecapable of executing software in association with appropriate software.When provided by a processor, the functions may be provided by a singlededicated processor, by a single shared processor, or by a plurality ofindividual processors, some of which may be shared. Moreover, explicituse of the term “processor” or “engine” should not be construed to referexclusively to hardware capable of executing software, and mayimplicitly include, without limitation, digital signal processor (DSP)hardware, network processor, application specific integrated circuit(ASIC), field programmable gate array (FPGA), read only memory (ROM) forstoring software, random access memory (RAM), and non volatile storage.Other hardware, conventional and/or custom, may also be included.Similarly, any switches shown in the figures are conceptual only. Theirfunction may be carried out through the operation of program logic,through dedicated logic, through the interaction of program control anddedicated logic, or even manually, the particular technique beingselectable by the implementer as more specifically understood from thecontext.

FIG. 3 comprises a flow chart of a further embodiment of the methodaccording to the invention. In this embodiment, several heuristics 110a, 110 b, . . . , 110 n are used. Different types of heuristics in allkinds of combinations yield a plurality of possible specificembodiments, following the same principle as stated above. Theprobabilities obtained from the respective heuristic probabilityassignment steps 110 a, 110 b, . . . , 110 n have to be combined 115into a single probability that is used in the global optimization 130.FIG. 3 also illustrates the optional post-processing step 135. Theresult of the post-processing step 135 (or of the global optimizationalgorithm 130, if no post-processing is performed) may be fed back tothe probability combiner 115 to be treated as an additional heuristic.The end result of the foreground estimate 150 may be fed back to theheuristics 110 a, 110 b, . . . , 110 n to update the parameters used bythe latter, where applicable.

Without loss of generality, FIG. 4 illustrates a further embodimentusing two heuristics: a color model 110 a and a body model 110 b. Thecolor model 110 a may be based on a known adaptive background colormodel. The “Vibe” model may be used for this purpose, but other knownmodels may be equally suitable. “Vibe” is a known technique which uses adynamic set of color samples at each pixel to represent the backgroundcolor distribution. As the model is not trained during an initializationphase, the method is adapted to incorporate a confidence measure. Atinitialization, this confidence is put to zero for all pixels, implyingthat no background color information is known at that point. Furtherimplementation details pertaining to “Vibe” can be found in the Europeanpatent specification EP 2 015 252 B1, entitled “Visual backgroundextractor”.

A stand-alone color model suffers from several drawbacks, including theneed for an initialization procedure, where no object-of-interest is inthe scope of the camera, only background, so the system can learn thebackground model. It would also eventually fade motionless foregroundinto the background because of the time-constant inherent to adaptivity.It is therefore advantageous to incorporate an optional secondheuristic. The illustrated embodiment uses a very generic body modelprobability based upon a face tracker.

The camera input feed 1 gets picked up by a face detector 5 (e.g.Viola-Jones, haarwavelet-classifier, . . . ). Because these detectorsmight produce false positives and in view of the noise on the positionand size of the face, the resulting face candidates get filtered andtracked 6 to produce a single good estimate of the face bounding box(i.e. position and size). Using face position and size obtained in thismanner, we transform 8 (mere translation and rescaling) a very genericbody probability map 7 to match the face in the input video feed. Thisacts as the basis for the first heuristic probability assignmentalgorithm 110 b for our foreground estimation.

A background color model 3 is used as the basis for the second heuristicprobability assignment algorithm 110 a. Various techniques exist, likebackground subtraction, Gaussian mixture models, or “Vibe”.

Each frame a segmenting module 2 compares the input video feed againstthis background model and outputs a foreground probability map. If theconfidence measure is low, the foreground probability will be close to50%. If a good match is detected between the color model and the inputfeed color of a pixel, its foreground probability will be close to 0,otherwise it is more likely to be foreground and the probability will becloser to 100%. If this is done for every pixel, this yields anotherprobability map.

The color model is preferably updated 4 on the basis of the final resultobtained after combining all probabilities and incorporating objectboundaries. The confidence measure for each pixel is also updated. Itincreases with newly learned color samples and decreases slowly if nocolor observation is made for a long time, posing a learning timeconstant, which will be compensated by a feed-back loop and the otherheuristic to prevent the fading to background problem.

The two probability maps 2, 8, together with a probability map resultingfrom the foreground segmentation of a previous frame 12 are fed into theprobability combination module 115. If no previous frame information isavailable, the foreground-background probability map is globallyinitialized at 50%. The probability combination may be a simplemultiply-add operation, or any other known operation for compositingprobabilities. The result is used as an input to the global optimizationmethod 130.

In the illustrated embodiment, object boundaries are determined by aSobel edge detector 120, which is a simple gradient method.Alternatively, known techniques such as Laplace edge detectors, Cannyedge detectors, and true segmentation methods may be used. This resultis also used as an input to the global optimization method 130.

The global optimizer 130 used in the invention may be implemented by anyof the known techniques, including Graph Cut, Belief Propagation, andtheir variants. The illustrated embodiment uses a bilateral filter 130as a global optimizer. The module approaches the foreground estimationproblem as a global optimization problem, which has to be solvediteratively, but locally. The input probability map obtained fromcombining the probability maps 115 acts as a data term, while the objectboundaries obtained from the Sobel edge detector 120 act as asmoothening term. At each iteration of the algorithm, and for eachpixel, the probability map is updated by a weighted combination of theprobabilities of its neighbors. These weights are large when no objectboundary is between the neighboring pixels and small otherwise. This wayhigh foreground probabilities at one pixel propagate over the imageuntil object boundaries are met. This is also true for low foregroundprobabilities (i.e. high background probabilities). It resemblescoloring between the object boundaries of the foreground probabilities,eventually saturating to a full 100% foreground or a full 100%background.

This bilateral filter module 130 can be implemented in a layeredmulti-level way to increase performance. Furthermore, it may implementdifferent kinds of probability combiners.

The output of the bilateral filter module 130 may still yield someartifacts around regions with a lot of object boundaries, because thereprobability propagation is countered a lot. It may therefore beadvantageous to apply some post-processing 135. In this embodimentmorphological operators 11 like dilution and erosion are used toeliminate these artifacts. Then a connected-component module 12 is usedto only select objects of a relevant minimal size. Tracking and similartechniques may also be applied. It will be clear to the skilled personthat various combinations of these techniques may be selected, dependingon the requirements of the specific application.

The final output can be reused as a foreground estimate 150 by theapplication, for instance to do background substitution. Also, it is fedback into the probability combiner 115, since foreground probability inone frame for a pixel increases the probability of foreground of thatpixel in the next frame. It is also this final result which is used bythe “Vibe” learning module 4 to update the color model 3. Samples atpixels labeled background are learned more quickly than regions labeledforeground.

The illustrated embodiment combines the advantages of two techniques.The generic body model will help bootstrap the learning function withinthe color model and keep doing so along the way. Because the learningtime constant may be a lot smaller in this case than in a stand-alonebackground model based technique, making it more adaptive to lightchanges, camera movement etc., it continues to require externalinformation from the other heuristics through the described system.

Other heuristics may be used to detect foreground, provided that theheuristic is based on an adequate definition of the “foreground” as theset of objects of interest. Different definitions result in differentembodiments. Some heuristics are especially good at segmenting imagescomprising people, but suitable heuristics may be defined to segmentimages comprising objects of any arbitrary type.

-   -   If foreground is defined as the set of objects being spatially        close to the camera, depth information can be used as a fairly        good heuristic. This depth information can come from processing        image-pairs from a stereo-pair, from time-of-flight cameras, and        the likes. Because this processing tends to be a lot less        accurate than needed for foreground segmentation (the method        being especially prone to labeling objects in the background as        foreground), feeding it through the global optimization        module—with or without combination with other heuristics—is        still needed.    -   In a human-detection oriented heuristic, we may define        skin-colored pixels as being foreground. A probability map may        thus be created with a lot of unknown 50% probabilities, but        with some high foreground probabilities as well at the position        of the face and the hands.    -   In a similar way, a foreground color model can be established in        a gradual way, incorporating for instance a pixel-dependent        and/or region-dependent histogram of colors occurring in the        foreground: skin colors, hair colors, clothing colors or        textures.    -   Foreground may also be defined as being moving pixels. A motion        estimation algorithm like optical flow can be used to indicate        moving pixels, which will get a high foreground probability.        Since a lack of motion does not necessarily mean that the region        in question is not foreground, pixels with no motion will get a        foreground probability slightly below 50%. This rough initial        classification may still yield good results thanks to the        feedback-loop in the system.    -   A background edge model can be built, wherein regions were edges        appear or disappear are indicated as foreground, leaving the        other regions at 50% probability.

As illustrated by the above examples, embodiments of the presentinvention may present the following advantages:

-   -   no need an initialization phase or a green screen;    -   fast stabilization when the camera is moved;    -   adaptiveness to lighting condition changes without the        disadvantage that foreground gradually fades to background;    -   use of more intelligent heuristics to find foreground, the way        the human visual system does;    -   explicit use of object boundaries as boundaries for the        segmentation.

A person of skill in the art would readily recognize that steps ofvarious above-described methods can be performed by programmedcomputers. Herein, some embodiments are intended to cover programstorage devices, e.g., digital data storage media, which are machine orcomputer readable and encode machine-executable or computer-executableprograms of instructions where said instructions perform some or all ofthe steps of methods described herein. The program storage devices maybe, e.g., digital memories, magnetic storage media such as a magneticdisks or tapes, hard drives, or optically readable digital data storagemedia. The embodiments are also intended to cover computers programmedto perform said steps of methods described herein.

The invention has been described herein by means of several exemplaryembodiments. These embodiments serve to illustrate but not to limit theinvention. It will be clear to the skilled person that featuresdescribed with respect to one embodiment, may be freely combined withfeatures described in other embodiments, to obtain the described effectsand/or advantages of said features.

1. A method for segmenting an image from a sequence of video images intoa foreground and a background, said image being composed of pixels, themethod comprising: assigning an initial foreground probability to eachof said pixels; assigning a set of probability propagation coefficientsto each of said pixels; and applying a global optimizer to the initialforeground probabilities of said pixels to classify each of said pixelsas a foreground pixel or a background pixel, to obtain a definitiveforeground map; wherein said global optimizer classifies each processedpixel in function of the initial foreground probability of saidprocessed pixel and the initial foreground probability of neighboringpixels, the relative weight of the initial foreground probability ofneighboring pixels being determined by the probability propagationcoefficients of said neighboring pixels.
 2. The method according toclaim 1, wherein said assigning of said foreground probability comprisesapplying at least a first heuristic probability assignment algorithm anda second heuristic probability assignment algorithm, and determiningsaid foreground probability by combining a result of said firstheuristic probability assignment algorithm with a result of said secondheuristic probability assignment algorithm.
 3. The method according toclaim 2, wherein one of said first heuristic probability assignmentalgorithm and said second heuristic probability assignment algorithmcomprises comparing said pixels to a background color model.
 4. Themethod according to claim 2, wherein one of said first heuristicprobability assignment algorithm and said second heuristic probabilityassignment algorithm comprises applying a human body model.
 5. Themethod according to claim 2, further comprising updating a parameter ofat least one of said first heuristic probability allocation algorithmand said second heuristic probability allocation algorithm on the basisof said definitive foreground map.
 6. The method according to claim 1,further comprising a post-processing for removing allocation artifacts.7. The method according to claim wherein said post-processing comprisesapplying morphological operators.
 8. The method according to claim 6,wherein said post-processing step comprises applying aconnected-component algorithm.
 9. The method of claim 1, wherein theglobal optimizer is an iteratively applied bilateral filter.
 10. Acomputer program product comprising computer-executable instructions forperforming the method of claim
 1. 11. A system for segmenting an imagefrom a sequence of video images into a foreground and a background, saidimage being composed of pixels, the system comprising: an inputinterface for obtaining the image; an assignment engine for assigning aninitial foreground probability to each of said pixels; a contourdetector for assigning a set of probability propagation coefficients toeach of said pixels; and a global optimizer for operating on the initialforeground probabilities of said pixels to create a definitiveforeground map by classifying each of said pixels as a foreground pixelor a background pixel; and an output interface to provide saiddefinitive foreground map; wherein said global optimizer is adapted toclassify each processed pixel in function of the initial foregroundprobability of said processed pixel and the initial foregroundprobability of neighboring pixels, the relative weight of the initialforeground probability of neighboring pixels being determined by theprobability propagation coefficients of said neighboring pixels.
 12. Thesystem according to claim 11, wherein said assignment engine is adaptedto apply at least a first heuristic probability assignment algorithm anda second heuristic probability assignment algorithm, and to determinesaid foreground probability by combining a result of said firstheuristic probability assignment algorithm with a result of said secondheuristic probability assignment algorithm.
 13. The system according toclaim 12, wherein one of said first heuristic probability assignmentalgorithm and said second heuristic probability assignment algorithmcomprises comparing said pixels to a background color model.
 14. Thesystem according to claim 12, wherein one of said first heuristicprobability assignment algorithm and said second heuristic probabilityassignment algorithm comprises applying a human body model.
 15. Thesystem according to claim 12, further adapted to update a parameter ofat least one of said first heuristic probability allocation algorithmand said second heuristic probability allocation algorithm on the basisof said definitive foreground map.